Skip to content

fix(flows): inherit usage_metadata when after_model_callback replaces a response - #7449

Closed
zhuhongd wants to merge 2 commits into
google:mainfrom
zhuhongd:fix/after-model-callback-usage-metadata
Closed

zhuhongd wants to merge 2 commits into
google:mainfrom
zhuhongd:fix/after-model-callback-usage-metadata

Conversation

@zhuhongd

@zhuhongd zhuhongd commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Link to Issue or Description of Change

1. Link to an existing issue:

Problem:

Follow-up to #7035 / #7036. A callback-built replacement now inherits
partial and turn_complete, but usage_metadata was left out of that
fix's scope and is still dropped. finalize_model_response_event copies
only non-None fields into the Event, so when a rebuilding
after_model_callback leaves usage_metadata unset, the yielded final event
and the persisted session event carry no usage for that model call. The
BigQuery analytics plugin's event path, the A2A converters,
agent_test_runner, and any per-call cost attribution from session history
all see nothing. Happens in streaming and non-streaming mode alike (repro in
the linked issue).

Solution:

Extend the existing inheritance in flows/llm_flows/core/_finalizer.py to
usage_metadata, with the same semantics as 39e4538: inherit only when the
replacement leaves the field unset, copy-on-inherit so the callback-owned
object is not mutated, and an explicitly set value always wins (a callback
that made an additional model call can report its own combined usage). The
helper is renamed _inherit_unset_model_fields to match its widened scope;
it is private and had no other references.

Scope is deliberately one field. grounding_metadata already has its own
handling in the same function, and finish_reason / error_code are left
to the callback because a guardrail replacement may legitimately change
finish semantics. Rationale for treating usage_metadata as model-owned:
it measures the call as billed by the provider, not the content, so there is
no reading under which None is the more accurate value after a content
rewrite.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added to tests/unittests/flows/llm_flows/test_base_llm_flow_partial_handling.py,
mirroring the tests from #7036:

  • test_after_model_callback_replacement_inherits_usage_metadata — a
    rebuilt replacement carries the original's usage; the callback's object is
    not mutated.
  • test_after_model_callback_replacement_explicit_usage_metadata_respected
    — a replacement that sets its own usage is not overridden.
  • test_run_async_sse_rebuilt_final_keeps_usage_metadata — flow-level SSE
    run with a fake model that reports usage on the final response only:
    events are partial == [True, True, None] and
    usage_metadata == [None, None, <usage>].
tests/unittests/flows/llm_flows/test_base_llm_flow_partial_handling.py
  14 passed (11 pre-existing + 3 new)

tests/unittests/flows/llm_flows/ + tests/unittests/agents/
  1846 passed, 2 xfailed

Manual End-to-End (E2E) Tests:

Ran the repro script from the linked issue (fake model, no API key) against
this branch. Before: REBUILD non-stream and REBUILD SSE both show
final_event.usage=None persisted.usage=None. After: all four runs show
usage=128, and the partial flags from #7036 are unchanged.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. (N/A)

Additional context

Formatted with isort + pyink per pyproject.toml (no changes needed).

… a response

A callback that returns a replacement LlmResponse rarely sets
usage_metadata. finalize_model_response_event copies only non-None fields
into the Event, so the yielded final event and the persisted session event
carry no usage for that model call: the BigQuery analytics plugin's event
path, the A2A converters and per-call cost attribution from session history
all see nothing.

usage_metadata measures the model call (tokens billed by the provider), not
the content of the response, so replacing the content does not change it.
Inherit it from the replaced response when the replacement leaves it unset,
alongside the partial/turn_complete inheritance from 39e4538. An explicitly
set value on the replacement is still respected, and the callback-owned
object is not mutated. Renames the helper to _inherit_unset_model_fields to
match its widened scope.

Follow-up to google#7035.
@adk-bot

adk-bot commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Thank you @zhuhongd for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit 8e386a9.

Closing this PR as the changes are now in the main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

after_model_callback: a replacement LlmResponse drops usage_metadata, erasing the model call from token accounting

3 participants